Benchmarking Memory Performance with the Data Cube Operator

Data movement across computer memory hierarchy and across hosts of distributed systems is known to be a limiting factor for applications processing large data sets. We use the Data Cube Operator on an Arithmetic Data Set, called ADC, to benchmark computer capability to handle large datasets. To compute the operator we implement a parallel algorithm that computes a view from the smallest parent. The algorithm employs RB-trees to process data fitting into memory and a multi-way merge to process data residing in secondary storage. The ADC stresses all levels of memory and storage by generating some of 2 views of an Arithmetic Data Set of d-tuples described by a small number of integers. Data intensity of the ADC can be controlled by selecting the tuple parameters, the sizes of the views, and the number of generated views. We present benchmarking results of memory performance of a number of computer architectures and of a small distributed system. Based on the benchmark we build a tool which reveals a computer memory signature and allows to rank computer memory performance.